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Abstract 

Design optimization of large systems can be attempted through a subproblem strategy. In this strat- 
egy, the original problem is divided into a number of smaller problems that are clustered together to ob- 
tain a sequence of subproblems. Solution to the large problem is attempted iteratively through repeated 
solutions to the modest subproblems. This strategy is applicable to structures and to multidisciplinary 
systems. For structures, clustering the substructures generates the sequence of subproblems. For a 
multidisciplinary system, individual disciplines, accounting for coupling, can be considered as subprob- 
lems. A subproblem, if required, can be further broken down to accommodate subdisciplines. The sub- 
problem strategy is being implemented into the NASA design optimization test bed, referred to as 
“CometBoards.” Neural network and regression approximators are employed for reanalysis and sensitiv- 
ity analysis calculations at the subproblem level. The strategy has been implemented in sequential as 
well as parallel computational environments. This strategy, which attempts to alleviate algorithmic and 
reanalysis deficiencies, has the potential to become a powerful design tool. However, several issues have 
to be addressed before its full potential can be harnessed. This paper illustrates the strategy and addresses 
some issues. 


Introduction 

Convergence difficulty can be encountered when nonlinear mathematical optimization methods are 
used to solve large multidisciplinary problems with many design variables and numerous behavior con- 
straints. Solution to such problems can be attempted through a subproblem strategy in which the original 
problem is replaced by an equivalent sequence of subproblems as shown in figure 1 . A subproblem with 
a few design variables and a small number of constraints can be solved with available nonlinear pro- 
gramming algorithms. Solution to the larger problem can be attempted by repeating solutions to the 
sequence of modest optimization subproblems until convergence is achieved for the original problem. 
This strategy is applicable to structures and to multidisciplinary systems. For structures, clustering sub- 
structures generates the sequence of subproblems. For a multidisciplinary system, individual disciplines, 
accounting for coupling, can be considered as subproblems. A subproblem, if required, can be further 
broken down to accommodate subdisciplines. 

The subproblem strategy is being implemented into the NASA design optimization test bed, 
referred to as “CometBoards” (ref. 1). Neural network and regression approximators are employed for 
reanalysis and sensitivity analysis calculations at the subproblem level. The strategy is available in 
sequential as well as parallel computational environments. This strategy, which attempts to alleviate 
algorithmic and reanalysis deficiencies, has the potential to become a powerful design tool. However, 
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Figure 1 . — Subproblem coupling. 


several issues have to be addressed before its full potential can be harnessed. This paper addresses and 
illustrates issues relating to analysis approximations and convergence to local solutions. The paper is 
presented in these subsequent sections: design test bed CometBoards; subproblem solution strategy; 
analysis approximation; illustrative example; convergence to local solutions; and conclusions. 


Design Test Bed CometBoards 

The test bed CometBoards, an acronym meaning comparative evaluation test bed of optimization and 
analysis routines for the design of structures, was originally developed to compare different nonlinear 
mathematical programming algorithms and different analysis methods for structural design applications. 
The test bed was subsequently expanded for multidisciplinary design problems. Its modular organization 
is shown in figure 2. The CometBoards system first formulates the design as a nonlinear mathematical 
programming problem, reading infonnation specified in input data files that include initial design with 
upper and lower bounds, limitation on behavior constraints, definition of subproblems, cascade strategy 
(ref. 2), and so forth. It then solves the resulting problem by executing the required segments of the 
program. 

Problem formulation can use several types of analysis tools available in its analyzer module. The 
analyzers available include LE_HOST (ref. 3) and COSMIC/NASTRAN (ref 4) for structural analysis, 
FLOPS (flight optimization system for aircraft analysis, ref. 5), NEPP (NASA Engine Performance Pro- 
gram for air-breathing engine cycle analysis, ref. 6), and others. Alternatively, a user-specified analyzer, 
which can be integrated through a soft coupling strategy, can be used. The solution to the optimization 
problem can use any one of a dozen optimization algorithms (ref. 7) available in CometBoards or can use 
a cascade strategy. 

The cascade strategy is created by combining more than one optimizer in a specified sequence with 
pseudorandom perturbation between two optimizers. An approximation module with neural network and 
regression methods available in CometBoards can be used to reduce the number of calculations in optimi- 
zation (refs. 8 and 9). The present version of CometBoards can accommodate several different disciplines, 
each of which can be further divided into subproblems. Thus, the CometBoards design tool can optimize 
a system that can be defined in terms of about 100 optimization subproblems. Alternatively, the Comet- 
Boards test bed can be used to examine the optimality of a small portion of a larger design problem by 
appropriate input data specification. 
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Figure 2. — Organization of CometBoards test bed. 


Subproblem Solution Strategy 

Subproblem solution strategies for sequential and parallel algorithms are depicted in the flow dia- 
grams of figure 3. The flow diagrams each have two loops. The inner loop (z = 1, AO is associated with 
the optimization of N subproblems. In the parallel computational environment the subproblems are dis- 
tributed to different processors. The outer loop repeats the solution of the N subproblems several times 
(referred to as “cycles”) until convergence. In the sequential computational environment the subproblems 
are optimized in sequence, and the design variables for the entire structure (also referred to as “global 
variables”) are updated as soon as a solution to any single subproblem is available. In parallel computa- 
tion, wherein subproblems are assigned to different processors, the global design variables can be updated 
only after the solutions have been completed for all the subproblems. In other words, the sequential algo- 
rithm, which benefits from intermediate improvement to the global design variables, can converge faster 
than the parallel algorithm because updating the global variables in this scheme cannot proceed until a full 
subproblem optimization cycle has been completed. 
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(a) Sequential calculations. (b) Parallel calculations. 

Figure 3. — Substructure solution strategy. 


Analysis Approximation 

Approximation is employed to simplify analysis and reduce distortion of the design space. The salient 
features of analysis approximation are as follows: 

1 . Analysis approximation is carried out for each subproblem prior to its solution. 

2. Objective function and the constraints are separately approximated. Provision exists to 
approximate a component of a constraint (such as the stress a instead of the stress constraint 
{g = |c/c 0 | - 1 < 0}). The approximation can be carried out either for the large number of raw 
constraints or after the constraint formulation. 

3. The approximation provides the value of the function and its gradient with respect to the design 
variables. 

4. Both the linear regression method and neural networks can be used for approximations. An 
approximate analyzer at a subproblem solution stage is formulated in the following steps: 

(a) Select the basis functions. 

(b) Establish a benchmark solution around which input-output pairs are chosen. 

(c) Generate good-quality input-output pairs for objective function and constraints or their com- 
ponents. 

(d) Train the approximate methods with validation. This process provides the constraints and 
their sensitivities. 
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(e) Use the approximators for the subproblem solution. 

(f) Repeat steps (b-d) prior to each subproblem solution. Step (a) has to be performed once for 
the entire optimization process. 


Linear Regression Analysis 

The linear regression method available in CometBoards approximates a function by using the follow- 
ing basis functions: (1) a cubic polynomial, (2) a quadratic polynomial, (3) a linear polynomial in recipro- 
cal variables, (4) a quadratic polynomial in reciprocal variables, and (5) combinations of these functions. 
The regression coefficients are determined by using the linear least-squares method in the LAPACK 
library (DGELS routine, ref. 10). The gradient matrix of the regression function with respect to the design 
variables is obtained in closed form. Once the regression coefficients have been obtained, the reanalysis 
and the sensitivity analyses require trivial computational effort. 


Neural Network Approximations 

The neural network approximation available in CometBoards is referred to as “Cometnet” (ref. 8). 

It approximates the function with a set of kernel functions. Cometnet permits approximations with linear 
and reciprocal polynomials, as well as with Cauchy and Gaussian radial functions. A singular-value 
decomposition algorithm is used to calculate the weight factors in the approximate function during net- 
work training. A clustering algorithm in conjunction with a competing complexity-based regularization 
algorithm is used to select suitable parameters for defining the radial functions. 


Benchmark Solution and Input-Output Training Pairs 

The initial design is considered as the benchmark solution for generating the input-output pairs for the 
first subproblem. For subsequent subproblems the benchmark solution is updated by using optimization 
solutions available at that iteration stage. For generating the input-output pairs to train the approximate 
methods, a set of design points is selected at random within their upper and lower bounds by using a 
design-of-experiment strategy. The number of input-output pairs and the bounds of the design variables 
are changed as the optimization iterations progress. 


Training the Approximate Methods 

Both the regression and neural network methods are trained for a set of good-quality input-output 
training pairs. The basis functions are made as identical as possible between the methods. Selection of 
similar basis functions allows a systematic comparison of the two approximation concepts. Consider, for 
example, substructure 1 of the trussed-ring problem subsequently described in this paper. The basis func- 
tions used in the regression method for this substructure consist of linear and quadratic polynomials in 
design variables and linear polynomials in reciprocal variables. For the substructure with eight design 
variables the basis function becomes a polynomial with 53 coefficients. Likewise, for the same substruc- 
ture the neural network scheme has 53 coefficients. In other words, the number of coefficients in the 
numerical approximations is the same for both the neural network and regression methods. 
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Figure 4. — Sixty-bar trussed ring. 


Illustrative Example 

The subproblem solution strategy is illustrated by considering the design of a modest trussed steel 
ring shown in figure 4, as an example. The ring is made of 60 bars with inner and outer diameters of 
180 and 200 in., respectively. It is fully restrained at node 10 and free to move only along the y direction 
at the diametrically opposite node 16. The ring is subjected to two load conditions. The first load condi- 
tion consists of a 40-kip compression along its horizontal diameter, applied at nodes 1 and 7. The second 
40-kip load, applied at node 4, induces compression along the vertical diameter. For the purpose of design 
the 60 bar areas of the truss are linked to obtain 16 independent design variables. Sixty stress constraints 
(with a yield strength of 20 ksi), for each load condition, are grouped to obtain 16 constraints. The distor- 
tions of the ring along the horizontal and vertical diameters are controlled through 4-in. displacement 
limitations specified at nodes 1 , 4, and 7 for each load condition. The ring is designed under problem 
1 and problem 2. In problem 1 the optimum weight of the ring is obtained for stress and displacement 
constraints. Problem 2 includes all the constraints of problem 1 and a frequency limitation at 14 Hz. 
Subproblem solution is obtained in the following steps. 
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Step 1: Substructure Model 


A substructure model for the ring is obtained by dividing it into four substructures. The substructures 
are described in table 1. Take, for example, substructure 1. It encompasses 12 bars that form the outer ring 
(see fig. 4). The 12 bar areas are linked to obtain four design variables. Likewise, the other three substruc- 
tures are defined in table 1 . 


TABLE 1.— DESIGN VARIABLES AND SUBSTRUCTURES OF RING PROBLEM 


Design 

variables 

Area of bars 

Substructures 

Remarks 

1 

1,2,3 

I 

Outer ring northeast (see fig. 4) 

2 

4,5,6 

I 

Outer ring northwest 

3 

7,8,9 

I 

Outer ring southwest 

4 

10,11,12 

I 

Outer ring southeast 

5 

13,14,15 

II 

Inner ring northeast 

6 

16,17,18 

II 

Inner ring northwest 

7 

19,20,21 

II 

Inner ring southwest 

8 

22,23,24 

II 

Inner ring southeast 

9 

25,26,27,37,38,39 

III 

Crossbars northeast 

10 

28,29,30,40,41,42 

III 

Crossbars northwest 

11 

31,31,33,43,44,45 

III 

Crossbars southwest 

12 

34,35,36,46,47,48 

III 

Crossbars southeast 

13 

49,50,51 

IV 

Radial bars east and northeast 

14 

52,53,54 

IV 

Radial bars north and northwest 

15 

55,56,57 

IV 

Radial bars west and southwest 

16 

58,59,60 

IV 

Radial bars south and southeast 


Step 2: Definition of Subproblems 

The four substructures are clustered together to obtain four subproblems, as defined in table 2. For 
example, subproblem 1 is obtained by combining substructures III and I. It has eight design variables and 
eight stress and six displacement constraints for problem 1 . One frequency constraint is added to 
obtain problem 2. Other subproblems are defined in table 2. 


TABLE 2.— DEFINITION OF SUBPROBLEMS FOR RING PROBLEM 


Subproblems 

Substructures 

Remarks 

1 

I and III 

Outer ring and crossbars: 

8 design variables; 8 stress and 
6 displacement (and 1 frequency) constraints 

2 

II and IV 

Inner ring and radial bars: 

8 design variables; 8 stress and 
6 displacement (and 1 frequency) constraints 

3 

II and III 

Inner ring and crossbars: 

8 design variables; 8 stress and 
6 displacement (and 1 frequency) constraints 

4 

I and IV 

Outer ring and radial bars: 

8 design variables; 8 stress and 
6 displacement (and 1 frequency) constraints 
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Step 3: Solution of Subproblems 


The four subproblems are solved in a sequence, one after the other. Solution to the first subproblem 
is initiated from the specified initial design. For solution of subsequent problems the initial design is up- 
dated by using available subproblem solutions at that solution stage. Approximate analysis and sensitivity 
analysis models with neural network and regression methods are developed at the beginning of each sub- 
problem design. Solution of all four subproblems represents one cycle. The cycling is repeated until satis- 
factory solution is obtained for the large problem. 


Results for Ring Problem 

Solution to each ring problem has been obtained for nine different cases. Summaries of results for 
the two problems are depicted in tables 3 and 4. The mean and the standard deviation are given for the 
design variables instead of their individual values. The nine test cases considered are described here. 

Case 1 . — The ring is designed as a single problem without the use of subproblem strategy or the ap- 
proximation concepts. COSMIC/NASTRAN is the analysis tool. A sequential quadratic programming 
(SQP) algorithm is the optimizer. This design is considered the benchmark solution and all other results 
are compared with it. This solution was verified through a three-optimizer cascade strategy (method of 
feasible directions (FD), followed by SQP and the same FD). The cascade results confirmed the bench- 
mark solution. The method of feasible directions produced a 0.5 percent error in the weight and a 0.9 per- 
cent mean-square error for the design variables. Mild infeasibility was noticed for a few constraints. 


TABLE 3.— SOLUTIONS FOR RING PROBLEM WITH STRESS AND DISPLACEMENT CONSTRAINTS 
(PROBLEM 1) 


Test 

Description 

Optimum 

Design variables. 

Number of 

Number of 

CPU time 

case 


weight, 

lb 


sq. in. 

active 

constraints 

cycles 
(number of 

to 

solution, 







subproblems) 

min 




Mean 

Standard 

deviation 




i 

Single problem; 

COSMIC/NASTRAN analyzer; 

SQP optimizer (benchmark solution) 

799.9 

3.15 

0.81 

4 stress, 
1 disp. 

KD 

42.0 

2 

Four mutually exclusive 
subproblems; COSMIC/NASTRAN 
and SQP 

1487.8 

4.64 

3.4 


Failed 


3 

Four coupled subproblems; 
COSMIC/NASTRAN and SQP 

799.9 

3.15 

0.81 

4 stress, 
1 disp. 

4(16) 

81.9 

4 

Four coupled subproblems; 
regression analyzer and SQP; 
constraint formulation active both at 

800.1 

3.15 

0.81 

3 stress, 
1 disp. 
(a) 

3(13) 



training and design stages 







5 

Four coupled subproblems; 
regression analyzer and SQP; 
constraint formulation passive at 
both training and design stages 

800.1 

3.16 

0.82 

4 stress, 
1 disp. 

3(13) 

169.4 

6 

Four coupled subproblems; 
regression analyzer and SQP; 
constraint formulation passive at 
training but active at design stage 

800.1 

3.16 

0.82 

4 stress, 
1 disp. 

3(13) 

170.5 

7 

Same as case 4, using neural 

800.1 

3.15 

0.81 

4 stress, 

3(13) 



network analyzer 




1 disp. 



8 

Same as case 5, using neural 

800.6 

3.16 

0.81 

4 stress, 

3(12) 



network analyzer 




1 disp 



9 

Same as case 6, using neural 

800.1 

3.16 

0.82 

4 stress, 

3(13) 

197.6 


network analyzer 




1 disp. 




“Infeasibility at a small fraction of 1 percent. 
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TABLE 4.— SOLUTIONS FOR RING PROBLEM WITH STRESS, DISPLACEMENT, AND FREQUENCY CONSTRAINTS 
(PROBLEM 2) 


Test 

case 

Description 

Optimum 

weight, 

lb 

Design variables, 
sq. in. 

Number of 
active 
constraints 

Number of 
cycles 
(number of 

CPU 
time to 
solution. 







subproblems) 

min 




Mean 

Standard 

deviation 




i 

Single problem; 
COSMIC/NASTRAN analyzer; 

SQP optimizer (benchmark solution) 

900.0 

3.52 

1.08 

8 stress, 

1 frequency 

HD 

80.5 

2 

Four mutually exclusive 

905.9 

3.54 

1.29 

5 stress. 

2(8) 

347.6 


subproblems; COSMIC/NASTRAN 
and SQP 




1 frequency 



3 

Four coupled subproblems; 
COSMIC/NASTRAN and SQP 

899.9 

3.50 

1.09 

7 stress, 

1 frequency 
(a) 

17 (68) 

203.2 

4 

Four coupled subproblems; 
regression analyzer and SQP; 
constraint formulation active both at 

859.8 


Infeasible at 55 percent and underdesign; failed 



training and design stages 







5 

Four coupled subproblems; 
regression analyzer and SQP; 
constraint formulation passive at 
both training and design stages 

900.8 

3.51 

1.07 

8 stress, 

1 frequency 
(a) 

3(13) 

288.9 

6 

Four coupled subproblems; 
regression analyzer and SQP; 
constraint formulation passive at 
training but active at design stage 

900.8 

3.51 

1.07 

8 stress, 

1 frequency 
(a) 

3(13) 

289.6 

7 

Same as case 4, using neural 
network analyzer 

859.8 



Failed 



8 

Same as case 5, using neural 

900.6 

3.51 

1.07 

8 stress. 

3(13) 

319.5 


network analyzer 




1 frequency 
(a) 



9 

Same as case 6, using neural 

900.8 

3.51 

1.07 

8 stress. 

3(13) 

320.9 


network analyzer 




1 frequency 
(a) 




Tnfeasibility at a small fraction of 1 percent. 


Case 2 . — The ring problem is solved by using the subproblem strategy and neglecting substructure 
coupling. The COSMIC/NASTRAN and SQP algorithms were used as the analysis and design tools, 
respectively. 

Case 3 . — This case is identical to case 2, but substructure coupling is included. 

Case 4 . — The ring problem is solved by using four coupled subproblems. Analysis is approximated 
by a linear regression method, and the SQP algorithm is used. Sixteen grouped stress constraints, the 
displacement constraints, and the weight are approximated. 

Case 5 . — Case 5 is the same as case 4, but approximation is carried out for all 120 stress constraints. 
All 120 stress constraints are used during design optimization. 

Case 6 . — Case 6 is the same as case 5, but the 16 grouped stress constraints are used during design 
optimization. 

Case 7. — Case 7 is the same as case 4, but linear regression is replaced by neural network 
approximation. 

Case 8 . — Case 8 is the same as case 5, but neural network approximation replaces the linear 
regression method. 

Case 9 . — Case 9 is the same as case 6, but linear regression is replaced by neural network 
approximation. 
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From the nine solution cases of problem 1 (see table 3) we observe the following: 

1 . The benchmark solution has an optimum weight of 800 lb. There are a total of five active 
constraints: four stress and one displacement. The mean value of the 16 design variables is 
3.15 square in. with a standard deviation of 0.81 square in. 

2. Out of the nine cases only case 2 (which excluded coupling between the substructures) failed 

to converge to the benchmark solution. The other eight cases produced acceptable solutions with 
minor variations. 

3. Typically, a subproblem strategy required three design cycles corresponding to 13 subproblem 
solutions. Subproblem solution strategy is numerically more expensive than regular optimization. 
With the COSMIC/NASTRAN analyzer the subproblem strategy is twice as expensive as the 
regular optimization. The regression and neural network approximators increase the burden by 
four and five times, respectively. The generation of the approximate models consumes the bulk 
of calculations. 

Likewise from the nine solution cases of problem 2 (which included a frequency constraint; 
see table 4) we observe the following: 

4. This benchmark solution has an optimum weight of 900 lb. There are a total of nine active con- 
straints: eight stress and one frequency. No displacement constraint is active. The mean value of 
the 16 design variables is 3.52 square in. with a standard deviation of 1.08 square in. 

5. Case 4 and case 7 failed to converge. Both cases trained a reduced number of stress constraints. 
Case 2, which failed earlier, was successful when a frequency constraint was included. Some 
variation is noticed in the number of active constraints for the nine cases. 

6. The inclusion of a frequency constraint increased the computational burden by a factor of 2 with 
regular analyzers and by a factor of 1.5 with the approximators. Regular optimization with and 
without the frequency constraint required 80.5 and 42.0 CPU minutes, respectively. Likewise, 
the inclusion of the frequency (see case 5) increased solution time from 170 to 289 CPU minutes 
for the regression model. The neural network method for the same case 5 was 10 percent more 
expensive than the regression method. A typical subproblem strategy required three design 
cycles that correspond to 13 subproblem solutions. 


Convergence to Local Solutions 

The subproblem strategy had a tendency to converge to local solutions different from that obtained 
when regular optimization is used. It was also observed that the variation in the minimum weight between 
the solutions was small, less than 1 percent. This mild variation in weight is within the numerical accu- 
racy of the algorithms. In other words, the minimum weight remained the same for the subproblem and 
regular optimization strategies. However, there was substantial variation in the design variables. 

The optimum depth profiles for a beam obtained for different combinations of behavior constraints 
and two solution methods are depicted in figure 5. The depth of the beam varied by about 10 to 15 percent 
between the two solution methods. The beam profile is more uniform for regular optimization than for 
subproblem optimization. Subproblem and regular solutions produced the same weight and the same set 
of active constraints. 
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Depth, in. Depth, in. Depth, in. Depth, in. 



[ (a) Optimum profile for stress, displacement, and frequency constraints. 



(b) Optimum profile for stress constraints. 



Figure 5. — Different optimum depth profiles for beam obtained with regular and substructure strategies. 
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Figure 6. — Optimum depth profile for cylindrical shell with regular and substructure strategies for stress and 
displacement constraints. 


The optimum depth profiles for a cylindrical shell obtained by using regular and subproblem strate- 
gies are depicted in figure 6. The profile generated from regular optimization is more uniform than that 
generated from substructure optimization. The optimum weights were 1 161.95 and 1 154.1 lb for the 
regular and substructure strategies, respectively. The difference of 0.676 percent can be considered 
negligible because of the complexity of the problem. The depth differed substantially between the two 
solutions. At the crown the optimum depths of 1.322 and 2.471 in. varied by 53 percent. At optimum the 
regular optimization and substructure strategies produced a different number of active constraints. 


Conclusions 

This study of subproblem solution strategy has led to the following conclusions: 

1 . Subproblem solution strategy has the potential to solve large problems. However, an adequate 
amount of coupling is required for a successful subproblem solution. 

2. The subproblem strategy can be numerically more expensive than regular optimization. 

3 . Both neural network and regression methods provide adequate analysis models and perform at 
about the same level of efficiency. 

4. Subproblem solution may produce an irregular design in contrast to regular optimization. Allevia- 
tion of this deficiency is important to the success of the strategy. 
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5. The training of the raw constraints instead of the formulated constraints requires additional 
research. 

6. A larger size and fewer subproblems can improve the efficiency of the strategy. 
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